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Abstract 

We calculate the contribution of anomalous top-quark interaction 
with a gluon, photon, and Z-boson (FCNC) to tt pair production 
in hadronic collisions. Using the current data from Run 1 of the 
Tevatron we evaluate the upper limit on the the anomalous coupling of 
t-quark with gluon, |re ff /A| < 0.52 TeV" 1 . We examine the production 
of double top (tt or tt) at hadronic colliders as a result of FCNC 
interactions of the top quark. It is shown that the study of such 
reactions at LHC collider makes possible to obtain the constraint on 
strength of coupling at the level of |a%/A| < 0.09 TeV -1 . 
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1 Introduction 



After the discovery of the top-quark at FNAL collider [Q, the new direction 
appeared in search for new physics beyond the Standard Model (SM). As 
the most massive quark in the SM, the t-quark is naturally considered to 
be more sensitive to new physics than lighter fermions. Therefore, precision 
measurements of the top quark properties and its production mechanisms 
provide unique possibility to obtain information on new physics. 

There are numerous speculations on possible new physics manifestations 
in the top quark sector. Of special interest is the study of top quark anoma- 
lous interactions due to flavour-changing neutral currents (FCNC) (see [ffl 
|, 1,0, 0,0). 



Indeed, the absence of elementary FCNC vertices in SM requires the con- 
sideration of loop contribution, which may generate the effective t — *> c(u) 
transitions ||. As a result, the top quark decays via FCNC are suppressed 
at the level of 10~ 10 to 10~ 12 , which makes their observation absolutely im- 
possible. 

Various extensions of SM lead to huge enhancement of such transitions. 
For example, one can compare the branching ratios of FCNC decays of the 
top quarks evaluating in the SM 0] and in MSSM H|: 

SM MSSM 
Br(t -> eg) « 10 -10 1(T 6 
Br(t -> 07) « KT 12 10 -8 
Br(t -> cZ) « HT 12 10 -8 

Therefore, any observation of FCNC decays would be an indication of a new 
physics beyond SM. 

The strategy of searching for anomalous top quark couplings consists 
of three complementary approaches: (i) obtain indirect bounds from low 
energy processes in which top quark anomalous couplings can enter via loop 
processes, or from the bound on t —>■ bW which puts limits on other decay 
modes of the top, (ii) direct searches at high energies for the effects of the 
anomalous couplings in top quark rare decays, and (iii) the study of the 
processes of top quark production via non-SM mechanisms. 

The search for rare top decays are extensively considered (see, for ex- 
ample, H |7]]). The experimental search for these decays was done by CDF 
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collaboration [Tj|. One of the possibility for study of the direct top pro- 
duction is considered in ||. In this scenario, a up (or charm) quark and a 
gluon from the colliding hadrons combine immediately to form an s-channel 
top quark, namely u(c) + g — > t. The consideration of other anomalous 



processes of the top production can be found in P, |12 . 

In this note we examine a manifestation of anomalous (FCNC)top inter- 
action in the process of top-antitop production: 

p(P)p tix, (l) 

as well as in the process of double top (or antitop) production: 

P(P)P -»• ttX. (2) 

The article is organised as follows. The anomalous FCNC interaction of 
t quark is considered in Section 2. The available constraints on anomalous 
couplings are considered in Section 3. In Section 4 we evaluate the upper 
limit on FCNC top interaction with a gluon from Run 1 of the Tevatron. 
The estimate for possible constraint on the same coupling, which may be 
reached at LHC collider, is given in Section 5. Section 6 presents general 
characteristics of the process of double (tt or it) top production in hadronic 
collisions. The study of this reaction and possible background processes are 
presented in Section 7. The main results are summarized in Conclusion. 

2 Anomalous ic Vertices 

Following |J, the vertices of the FCNC transitions t — > gc, t — > 7c, and 
t — > Zc can be written as follows: 

tgc gs^ia^[g L P L + g R P R ]t a cG a ^, (3) 

tic e^-ia^[ lL P L + lR P R }cF»\ (4) 

tZc => k z \ ~ i ltl [z L P L + z R P R }cZ^ (5) 
sin zt/w 

where A is the new physics cutoff, $w is the Weinberg angle, Pl,r = \(^Tl 5 ), 
ai iv _ -(ry^v _ j"^^ ^ ^ anc i ^ define the strength of the anomalous 
couplings for the currents with a gluon (n g ), photon (k 7 ), and Z boson (n z ), 
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respectively. The magnitudes of left and right components of the currents 
are denoted by gL,R, Jl,r, z l,r- They obey the obvious constraints: 

g 2 L + g R = l, 7 2 + 7 2 =l, zf + 4 = l. 

We also assume that lm.K g = Im/€ 7 = lm.K z = Im 7 j = Img^ = ImZi = 0, 
see [|, |]. 

Generally speaking, to avoid unitarity violation, the anomalous couplings 
k should depend on momentum transfer q 2 . For example, this dependence 
may have a form-factor-like behaviour 

K 2 K 2 K 2 

=► g ' 7 , , 4^^^. (6) 
A 2 A 2 (l + ^) l + 

However, such a dependence should be essential at very high values of mo- 
mentum transfer (> 1 TeV), and we ignore such dependence in the present 
article. 

Using the expressions for the vertices we find the equations for the 

corresponding widths (see also [§, §, [l(J): 

nt^cg) = tarf(jj.y (7) 

r(t^c 7 ) = «m t 3 ^, (8) 

r(t^cZ) = r} 2 mffl-4Vfl + 2^|Y (9) 
v ; 8 sin 2 2tJ w M 2 Z t \ m 2 t j \ m 2 t J 1 K J 

where a and a s are the QED and QCD coupling constants, respectively, and 
Mz is the mass of Z boson. 

In what follows we set the masses of the light quark (c or u) to zero, 
fn c = my. = 0, and 

m t = 175 GeV. 

We also assume that the anomalous couplings of top quark with c and u 
quarks are equal in magnitude. 

The corresponding values of the widths and branching ratios for different 
decay channels are given in the Table 1. 
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3 Current Constraints on \n g \ and |« 7 | 

The analysis of low energy processes (such as Kl — > Kl — Ks mass 

difference, 5° — _B° mixing, i? — > l + l~X, b — > 37, etc. (see, for example ||. [7| 
and references therein) results in the following constraints: 

\k b /A\ < 0.95 TeV~\ 
|« 7 /A| < 0.28 TeV -1 , 
\k z \ < 0.29. 

CDF collaboration performs the search for decays t — * 7 c("u) and t — > 
Zc(u) at FNAL collider in the reaction of — > ttV at the energy of 
^/s = 1.8 TeV. They obtained the upper limits on branching fractions for 
the decays t — > 70(11) and t — > Zc{u) |13| as follows: 

Br(t -> c 7 ) + Br(t -> ii 7 ) < 3.2% (95% CL), (10) 
Br(t -> cZ) + Br(t -> uZ) < 33% (95% CL). (11) 

Using the equations (§) and @, one can easily evaluate the upper limits 
on the constants of k 7 and k z (at m t = 175 GeV) from the experimental 
constraints ( fL0|) and (|TT|): 

K/A| < 0.77 TeV" 1 , 
K| < 0.74. 

Additional experimental constraints may be obtained from the study of 
the reaction e + e~ annihilation [|l(J 

e+ e~ -> 7*(Z*) -> t c (13) 

Starting from Summer 1997, the e + e~ collider LEP-2 operates at the energy 
oi y/s = 184 GeV (1997) and 189 GeV (1998). At this in the reaction (0) 
becomes kinematically possible, and therefore it will be possible to improve 
significantly the upper limits on the anomalous couplings of top interaction 
(see [|10| for details). 



4 Constraint on \n g \ from Run 1 of Tevatron 

In this section we consider what limit on anomalous coupling \k 9 \ can be 
derived from the available data on the cross section for ti pair production at 
Run 1 of the Tevatron. 
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In order to obtain a constraint on K g , we analyse the contribution of 
anomalous top interaction into tt production at the FNAL collider. 

Indeed, both CDF and D0 collaborations measured the top quarks pro- 
duction in bW final states. For the case, when any non-SM top interaction 
are very small, one has: 

Br(t — Wb) « 1 and a% p = a| M . 

However, for large values of the anomalous couplings the branching ratio to 
Wb final state may be substantially smaller than 100% (see Table 1). At the 
same time, there appeared new subprocesses leading to additional sources of 
the tt production. In this case the observed cross section (in the bbW + W~ 
final state) should be equal to: 

a(bW + bW~) = Br 2 (£ -> bW) x a(SM + New Physics). (14) 

Therefore, the difference between observed cross section for ti pair pro- 
duction and that calculated within (SM+New Physics) framework may be 
used to estimate upper limits on strength of anomalous coupling. 

To obtain the corresponding constraints on the anomalous couplings we 
require that 

\a% p - a(bW + bW-)\ < 2 A, (15) 

where A is an error consisting of both experimental and theoretical uncer- 
tainties. 

In the lowest order of perturbative QCD the tt production proceeds via 
quark-ant iquark and gluon-gluon annihilation (see Fig. la and lc). The 
anomalous top interactions lead to the appearance of new processes in which 
tt can be produced. The lowest order diagrams corresponding to these pro- 
cesses are presented in Fig. lb and Id. 

In further analysis we assume that other non-standard decay channels of 
the top are negligible. 

The production cross section measured by the CDF collaboration with an 



integrated luminosity of 110 pb 1 is a = 7.6^}'^ pb for m t = 175 GeV [pi 



which combines dilepton, lepton + jets and all-hadronic channels. The D0 
collaboration gives a = 5.9 ± 1.7 pb for m t = 172 GeV In our analysis 



we use the combined result 16 of 



erf p = 6.7 ± 1.3 pb. (16) 
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For the theoretical cross section in the SM, we adopt the most complete 



result currently available [17], which includes soft-gluon summation up to 



the next-to-leading logarithmic order: 

4 M = 5.06^pb (17) 

for m t = 175 GeV at ^ = 1.8 TeV. 

To make a comparison of the measured (<% p ) and calculated cross section 
(a(bW + bW~)), one should take into account the present uncertainty of the 



top quark mass ||16|| , which affects both the experimental and theoretical 
cross sections. Shifting m t by ±5 GeV changes the theoretical cross section 
by about ±15%. The measured cross section also changes with mt in the 
same direction, but the dependence is weaker. 

Combining all these uncertainties, the possible new physics contribution 
to the cross section is found to be 



A = + pb, 



where we set JE4|, [TJ] Acr cxp = 1.3 pb (see flUf )), and [jl7 

Aa th = (0.15 -=-0.20) a| M . 

In our case the inequality ( |15|) depends essentially on two parameters, K g 
and k z . The contribution due to a photon exchange (~ « 7 ) is very small. 

Bearing in mind different notations and normalizations used in the liter- 
ature, we present the evaluated constraints on anomalous constants in terms 
of constraints on corresponding branching ratios for top quark decays. 

The resulting constraints on the branching fractions Br(£ — > g{c + u)) 
and Br(t — > Z(c + u)) are shown in Fig. 2. The dashed band in this figure 
corresponds to two choices of Br(t — > 7(0 + u)) = 0% and Bx(t — > 7(0 + 
u)) = 3.2%. The maximum allowed values of these branching ratios are the 
following: 

Br(t -> g(c + u)) < 20% (at Br(t -> Z(c + u)) = 0), (18) 
Br(*-> Z(c + u)) < 32% (at Br(t -> g{c + u)) = 0). (19) 

These constraints correspond to the maximum allowed value of K g (at 
k z — ► 0) as follows: 

\k b /A\ < 0.52 TeV" 1 . (20) 
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We conclude that the variation of a t i within given uncertainty 2A does not 
allow to improve the upper limit on branching fraction to Z-boson channel. 
On the other hand, from Run 1 data we obtain the lowest current constraint 
on FCNC top quark interaction with a gluon. 

The differential distributions of top quarks produced in the reaction 

p p — ► tt X 

at ^/i = 1.8 TeV are shown in Fig. 3. It is seen from these pictures that at 
the FNAL collider energies the FCNC interaction gives practically the same 
form of the differential distributions of the top quarks like QCD calculations. 

5 Possible Constraints on \K g \ and \k z \ from tt 
Production at LHC 

Contrary to the case of the FNAL collider, the contribution of FCNC top 
quark interaction leads to significant modification of differential spectra at 
LHC energies. Fig. 4 presents the corresponding distributions of t quarks 
produced in the reaction 

p p —>■ tt X 

at y/s = 14 TeV. In fact, the FCNC interaction results in significant increase 
of tt production with high invariant mass M t i (see Fig. 4). Therefore, the 
observation of such deviation from QCD prediction could be considered as a 
signal from anomalous top quark interaction^]. 

At the same time, the value of this effect depends on the magnitudes of 
the anomalous couplings, K g , k z , and n 1 . Indeed, for anomalous coupling 
values about ~ 10~ 2 4- 10~ 3 this effect becomes very small. Therefore, the 
reaction of tt production may be not a good method for search for FCNC 
top quark interaction, 

Nevertheless, we estimate the possible limits on the anomalous coupling 
at LHC by using the same inequality (j!5l) . For the difference of the cross 

1 To be more precise, the shape of these distributions is determined by the point-like 
behavior of the effective couplings K (see explanation of the equation (|^)). However, for 
very high value of A (say, A > few TeV) we can ignore g 2 -dependence of k, and our 
calculated distributions remain the same. 
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sections calculated in SM and SM+FCNC frameworks (the A parameter) we 
use: 



Alhc = V - 15 ^ + °- 15 L P x 4 M « 175 pb. (21) 

The corresponding constraint on Br(£ — > g(c + u)) (versus Br(t — Z(c + 
u))) is shown in Fig.5. The maximum allowed value of this branching ratio 
is about 

Br(i -> g{c + u)) < 27%. (22) 

The corresponding maximum allowed value of K g (at k z — > 0) is: 

|/%/A| < 0.64 TeV^ 1 . (23) 

Note that this constraint on strength of coupling with a gluon is practically 
the same as that evaluated from the data of Run 1 of the Tevatron. Naturally, 
the obtained values of the upper limits (|22| ) and (|23|) are determined by our 
assumption on A L hc form (pl|). 

6 Double Top Production at Hadronic Col- 
liders 

The FCNC processes result in appearance of the new processes of the top 
quarks production: 

g u(c) -> t 

g u(c) -> t Z(7,</) 

g q — > t c(-u) 

g g -> tc{u) 



These are the processes of direct top production, considered, for example, in 
[§, p. Note, however, that all these processes have huge Q CD-background 
from W + n(jet) production, as well as from tt production. 
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In this article we examine the process which has no background from 
usual ti production. Namely, we investigate the process of double top (or 
antitop) production: 

p p -»• tt(ii) X. (24) 

Many processes may result in inclusive production of tt (it) pair in hadron- 
hadron collisions. For example: 

• 99 tttt 

• tt-^tt 

• dd^ tW~tW~ 

• ud^ g*W* -> tttb 
etc 

However, most of these reactions are highly suppressed by higher orders of 
strong and/or electro-weak coupling constants. Other suppressions result 
from specific kinematics of the production processes. The non-top final par- 
ticles should be produced in 'invisible' region for detector, i.e. they should 
escape into very forward/backward direction. 

On the other hand, the tt (it) pair from the reaction (^4j) may be produced 
in a simple 2 —>■ 2 subprocess with FCNC interaction. The corresponding 
diagrams describing tt production via anomalous top quark interaction are 
shown in Fig. 6. 

Naturally, to distinguish this process from conventional ti pair production 
one should require that both t-quarks decay semileptonically. In other words, 
we consider the reaction with two like-sign isolated leptons in the final state. 

We notice two advantages of the reaction (|24|): 

• it has no background from usual ti production (because ti pair decays 
into l + l~ pair) 

• to detect such a reaction one can apply the same methods as for the 
case of it production. 



The total cross sections for the reaction (p4j) at y/s = 14 TeV, m t 
175 GeV, and K g = k 7 = n z = 1 are equal to: 

a(tt) w 480 pb, 
o(E) w 16 pb. 
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Note that for this choice of the anomalous coupling the relative contributions 
due to gluon, photon, and Z-boson exchange are 



g : 7 : Z « 70 : 5 : 20. 

These values of the cross sections correspond to the following number of 
events (for the total integrated luminosity of 10 5 pb _1 and 100% efficiency 
of the isolated lepton detection): 

N(l + l + jet jet) l=ll<e w 1.9 ■ 10 6 , 
N(l-l-jetjet) l=lhe w 6.4 • 10 5 . 



Requiring the observation of at least one event with l + l + + 2-jets in the 
final state, we can estimate the value of the anomalous coupling k 9 which 
could be reached at LHC (at k 7 = k z = 0): 

\k 9 /A\ < 2.7- 10~ 2 TteV" 1 . 

This value is comparable with that obtained from the study of the reaction 
of the direct top production in the subprocess of gu — > c (see [|J). 

The differential distributions for the t quark from the reaction ([24]) are 
shown in the Fig. 7. Like for the case of tt production, the FCNC interaction 
results in significant increasing of tt production with a high invariant mass 
of M tt (see Fig. 7). 



7 Signal and Background Calculations 

Previous estimate was obtained in pure 'ideal' case (i.e. 100% efficiency for 
detection of leptons and jets and we assume no background process). Now 
we proceed to more 'realistic' estimates. 

The major sources of background to the tt production are 

qq' -> tiW ± (25) 
qq -> W ± q'W ± q' (26) 

The typical diagrams for each background process are shown in Fig. 8 (see 



Fig. 8a for the process Q2S) and Fig.8b for the process (p6|), respectively) 
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The total cross sections for these background processes are (at y/s = 
14 TeV) 



a(W + tt) = 0.5 pb, a{W~tt) = 0.24 pb, ( 
alw+W+qq) = 0.5 pb, a(W~W-qq) = 0.23 pb. ^ ' 

In order to simulate the detector effects in identifying the signal, we made 
a series of standard cuts on the transverse momentum (energy) p T (E T ), pseu- 
dorapidity 77, and the jet-lepton separation AR. We call them the 'standard' 
cuts: 

• p l T > 15 GeV, |^| < 2.5 

• ARij > 0.4 

• E{ et > 20 GeV, \rjjl < 2.5 

• two isolated like-sign leptons 

We take a standard b-tagging with the efficiencies for 6-quarks, charmed 
quarks, and light partons (u, d, s quarks and gluon) as follows: 

e 6 = 36%, e c = 10%, e uAs = 1%. 

We use the PYTHIA 5.7 + JETSET 7.4 Monte Carlo with addition of 
the routines calculating the signal and background processes. The result of 
the application of 'standard' cuts is shown in the Table 2. For the sake of 
comparison, we started from equal number of events {N start = 10 4 ) for each 
reaction, fl2§, fl2§), and (g§. 



The resulting distributions are shown in Fig.9-12. From these figures one 
may deduce the difference in the signal and background. 

First, in the signal the average number of jets is smaller than in back- 
ground from tiW (see Fig. 9). Both signal and background have jets with 
E-j* greater than 40-60 GeV (see Fig. 10). The invariant mass of the isolated 
charged lepton and jets from 'true' combinations (i.e. those coming from 
the decay of the same top quark) can not exceed the top mass (compare 
Figs. 11a and lib). The invariant mass of the two isolated charged leptons 
and two jets from the signal is much higher than that for background events 
(see Fig. 12). 
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In order to further enhance the signal with respect to background we 
impose the following additional cuts: 

"two jets" we require exactly two jets with u Ej et > 20 GeV; 
"Eq-" the energy of these two jets should be greater than 

40 GeV, and at least one jet should be tagged as a 6-jet; 
"M(Z j)" at least in one combination of the lepton+jet pairs both 
pairs should have invariant masses smaller than 160 GeV 
(i.e. Mihji) < 160 GeV and M(l 2 j 2 ) < 160 GeV); 
u M(l I j j)" the invariant mass of two charged leptons and two jets 
should be greater than 500 GeV. 
The result of application of all these additional cuts (step-by-step) are 
shown in the Table 2. One may deduce that applying all these cuts provide 
almost complete suppression of the background (even at high luminosity 
option). 



8 Discussion and Summary 

We may use the results of the signal and background calculations to deter- 
mine the maximum allowed value of K g /A that could be reached at LHC 
collider. We use the following criterion 

N s > 3.Jn s + N B , 

where Ns and Nb are the number of signal and background events, respec- 
tively. This criterion corresponds approximately to the 95% confidence level. 

For the total luminosity of / Cdt = 100 fb _1 we have Nb = 2.7 (for jijj, 
mode) and Nb = 10.8 (for 11 — /i/i + /ie + ee) (see Table 2). As a result we 
obtain (for k z — k 7 — 0) 

\k 9 /A\ < 0.116^(0.091,,) TeV-\ (28) 
Br(t^g(c + u)) < 1.2 x 10~ 2 ^(7A x 10- 3 ,,). (29) 

In summary, we calculate the contribution for tt pair production resulted 
from the anomalous top quark interaction (FCNC). From the analysis of the 
data from Run 1 of the Tevatron we deduce the lowest current limit on the 
anomalous coupling /%/A < 0.52 TeV -1 . 
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We investigate the double top-quark production at LHC collider via the 
chromo-magnetic flavor-changing neutral current couplings teg and tug. We 
find that the strength of the anomalous coupling iqg may be probed to 
K g f A = 0.09 TeV" 1 at the LHC with 100 fb -1 of data at y/s = 14 TeV. 
Assuming k 9 = 1, the scale of new physics A can be probed to 11 TeV at the 
LHC. 
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Table 1. The values of the anomalous couplings and corresponding branch- 
ing fractions for top quark decay, calculated within SM framework and by 
using the values resulting from the Tevatron Run 1 data and from low-energy 
data. The mass of the top quark is m t = 175 GeV, k/A is in TeV -1 . 





channel 
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qg 
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qZ 


SM 


k/A 




8 • 10~ 6 


3 ■ 10- 6 
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Br 


1. 


5- 10- 11 


5 • 10- 13 
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k/A 




1 


1 


K z = 1 




Br 


0.35 


0.32 


0.02(0.05) 


0.32 


FNAL 


k/A 




0.5 


0.77 


k z = 0.74 




Br 


0.57 


0.13 


0.018 


0.28 


Low-energy 


k/A 




0.5 


0.28 


k z = 0.29 




Br 


0.77 


0.17 


0.003 


0.056 



Table 2. Result of the application of differential cuts for the processes 
of tt production and relevant background reactions (see Section 7 for detail 
explanation). 
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Figure 1: Lowest order diagrams describing tt production in QCD 
(a and c) and in FCNC (b and d). 
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FNAL Vs = 1 .8 TeV 2A = 3.35 pb 
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Figure 2: Upper limit on Br(t — > g(c + it)) versus Br(i — > Z(c + 
u)) resulted from the analysis of the tt production at Run 1 of the 
Tevatron. 
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Figure 3: Differential distributions over pseudorapidity (rj), t quark 
energy (Et), transverse energy of the top (Et), and invariant mass of 
the ti pair (M tt ), for t quark produced in the reaction pp — ► iDT at 
s/s = 1.8 TeV (FNAL collider). Solid curves are QCD predictions, 
dashed curves are QCD+FCNC calculations. All distributions are 
given in arbitrary units. 
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Figure 4: The same distributions as in Fig.3, but calculated for the 
reaction pp — > ttXat v / i=14 TeV (LHC collider). 
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Figure 5: Upper limit on Br(i — > g(c + it)) versus Br(t — ► Z(c + u)) 
which may be obtained at LHC collider from analysis of the ti pair 
production cross section. 
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Figure 6: Diagrams describing tt quarks production due to FCNC 
interaction. 



22 




Figure 7: Differential distributions on pseudorapidity (77), t quark 
energy (Et), transverse energy of the top quark (Et), and invariant 
mass of the top-top pair (M tt ), for t quark produced in the reaction 
pp — > ttX at y/s = 14 TeV (LHC collider). All distributions are given 
in arbitrary units. (For comparison we show at the same plots the 
QCD predictions for the case of tt production at the same energy). 
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Figure 8: Diagrams describing the background processes for the 
reaction of tt pair production. 
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Vs = 1 4 TeV p p -> 1 1 X 
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Figure 9: Distributions on number of jets Nj et . 
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Figure 10: Ej -distributions from the signal (tt) and background 
processes. Ej* is in GeV. 
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Vs = 1 4 TeV p p -> 1 1 X 
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Figure 11: Distributions on invariant mass of charged lepton and 
jet. All combinations of leptons and jets are taken. M(fijet) is in 
GeV. 
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Vs = 1 4 TeV p p -> 1 1 X 
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Figure 12: Distributions on invariant mass of two charged leptons 
and two jets. M(/x ji jet jet) is in GeV. 
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